Clustering large data sets described with discrete distributions and its application on TIMSS data set

Symbolic Data Analysis is based on a special descriptions of data – symbolic objects. Such descriptions preserve more detailed information about data than the standard representations with mean values. A special kind of symbolic object is also representation with distributions. In the clustering process this representation enables us to consider the variables of all types at the same time. We present two clustering methods based on the data descriptions with discrete distributions: the adapted leaders method and the adapted agglomerative hierarchical clustering Ward’s method. Both methods are compatible – they can be viewed as two approaches for solving the same clustering optimization problem. In the obtained clustering, the leader is assigned to each cluster. The descriptions of the leaders offer simple interpretation of the clusters’ characteristics. The leaders method enables us to efficiently solve clustering problems with large number of units; while the agglomerative method is applied on the obtained leaders and enables us to decide upon the right number of clusters on the basis of the corresponding dendrogram. Both methods were successfully applied in analyses of different data sets. In the paper an application on the TIMSS data set is presented. The University of Ljubljana, Faculty of Economics, Department of Statistics, [email protected] (corresponding author) University of Ljubljana, Faculty of Mathematics and Physics, Department of Mathematics, [email protected] The Educational Research Institute, Slovenia, [email protected] 1 descriptions with distributions enable us to combine two data sets: answers of teachers and answers of their students, into one data set. The descriptions of the obtained clusters enable us to interpret the results in more understandable way.